Multi-timescale Pmscs for Music Audio Classification

Principal mel-spectrum components (PMSCs) [3] are computed several timescales in parallel [2]. For each timescale, the feature extraction involves four steps: discrete Fourier transform (DFT), mel-scaling, principal component analysis whitening (PCA) and temporal pooling. Firstly, for each timescale, we compute discrete Fourier transforms over a given time length. To compute PMSCs at different timescales, we simply use different time length for the DFT. In this system, we use a combination of 5 timescales: 46ms, 93ms, 186ms, 372ms and 743ms. We use the same frame step of 23ms for all timescales, meaning that there is more overlap for longer timescales. Secondly, we run the spectral amplitudes of the DFTs through a set of 200 mel-scaled triangular filters to obtain a set of spectral energy bands. We take the logarithm of the amplitude of those bands. Then, we compute the principal components of a random sub-sample of the dataset (roughly 60,000 frames). In order to obtain features with unitary variance, we multiply each component by the inverse square of its eigenvalue, a transformation known as PCA whitening. Here, the goal of the PCA is to diagonalize the covariance matrix, not to reduce dimensionality. Thus, we keep all the principal components (yielding 200 dimensions per timescale). The PCA whitened mel-scaled energy bands are referred to as PMSCs. In the next step, we apply temporal pooling, i.e. we compute statistics over the PMSCs over a given window length. We chose a time window of roughly 1.5 seconds. Within this time window, we compute the mean, the variance, the minimum and the maximum of each feature. Thus, for each timescale and for each 1.5 seconds window, we obtain 4 ∗ 200 = 800 features.